A Monte Carlo Analysis of the Fisher Randomization Technique: Reviving Randomization for Experimental Economists

Data created in a controlled laboratory setting are a relatively new phenomenon to economists. Traditional data analysis methods using either parametric or nonparametric tests are not necessarily the best option available to economists analyzing laboratory data. In 1935, Fisher proposed the randomization technique as an alternative data analysis method when examining treatment effects. The observed data are used to create a test statistic. Then treatment labels are shuffled across the data and the test statistic is recalculated. The original statistic can be ranked against all possible test statistics that can be generated by these data, and a p-value can be obtained. A Monte Carlo analysis of t-test, the Mann-Whitney U -test, and the exact randomization t-test is conducted. The exact randomization t-test compares favorably to the other two tests both in terms of size and power. Given the limited distributional assumptions necessary for implementation of the exact randomization test, these results suggest that experimental economists should consider using the exact randomization test more often.